Agent assist call transcription

An agent assist bot listens to a call using its own speech-to-text service. This transcript can be relayed to the corresponding regular call – the call that the customer conducts with the AI Agent – and used to replace that call’s own transcript.

This is primarily useful with speech-to-speech (realtime) models, whose user-side transcription is sometimes unreliable. By running an agent assist bot on the same call, you obtain a more accurate transcript from a dedicated speech-to-text service and use it in place of the realtime model’s transcription.

The relayed transcript replaces the regular call’s transcript before Post Call Analysis runs and before the conversation history is stored. As a result, post-call summaries, data extraction, the stored conversation history, and the finish webhook all reflect the agent assist transcript rather than the model’s unreliable transcription. The live conversation is not affected, and neither are the real-time user / llm webhooks sent during the call (these carry the model’s own transcription).

Configuring transcript relay

Configure the relay on the agent assist agent (the one that listens to the call). Set the agent_assist.transcript advanced configuration parameter:

user – replace only the user (customer) side of the regular call’s transcript.
both – replace both the user (customer) and the agent (bot) sides.
none – feature disabled (default).

Transcription mode

Setting agent_assist.transcript on its own runs the bot in a dedicated transcription mode (agent_assist.mode = transcript). In this mode the bot:

plays no welcome and never invokes the LLM – exactly like a passive assist bot – so no LLM interaction happens just to transcribe the call, and
requests speech recognition only for the side(s) it relays: user recognizes the customer only, both recognizes both sides. This avoids running speech-to-text on a side whose transcript is not used.

For convenience, agent_assist.mode and agent_assist.transcript imply each other:

configuring transcript alone implies mode = transcript;
configuring mode = transcript alone implies transcript = user.

You can also relay the transcript from a passive or active assist bot by setting both agent_assist.mode and agent_assist.transcript. In that case the bot keeps listening to all participants (as its mode requires) and, in addition, relays the configured transcript side(s).

To configure transcript relay:

Configure an agent assist bot on the call, as described in Passive agent assist mode. Make sure to select the speech-to-text service you want to use for the transcript.
Navigate to the 'AI Agents > Agents' screen, locate the agent assist agent, and click Edit.
1. Switch to the Advanced tab and add the agent_assist.transcript parameter. For example, to replace only the user side:
```
{
    "agent_assist": {
        "transcript": "user"
    }
}
```
2. Click Update.